Skip to content

fix: fix the system message in simple-chatbot example #1394

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 8 commits into from
Sep 30, 2025
Merged

fix: fix the system message in simple-chatbot example #1394

merged 8 commits into from
Sep 30, 2025
+8 −1

Conversation

yukuanj
Copy link
Contributor

@yukuanj yukuanj commented Sep 24, 2025

In the simple-chatbot example, I have updated the system message to explicitly suppress the triple backticks fences output by the LLM. Otherwise, the triple backticks fences would prevent the tool calling.

Motivation and Context

When I executed the simple-chatbot example, I observed:

python main.py
You: What tools do you have?
2025-09-24 17:31:02,549 - INFO - HTTP Request: POST https://api.groq.com/openai/v1/chat/completions "HTTP/1.1 200 OK"
2025-09-24 17:31:02,552 - INFO - 
Assistant: I have the following tools:

1. `read_query`: Execute a SELECT query on the SQLite database
2. `write_query`: Execute an INSERT, UPDATE, or DELETE query on the SQLite database
3. `create_table`: Create a new table in the SQLite database
4. `list_tables`: List all tables in the SQLite database
5. `describe_table`: Get the schema information for a specific table
6. `append_insight`: Add a business insight to the memo
7. `puppeteer_navigate`: Navigate to a URL
8. `puppeteer_screenshot`: Take a screenshot of the current page or a specific element
9. `puppeteer_click`: Click an element on the page
10. `puppeteer_fill`: Fill out an input field
11. `puppeteer_select`: Select an element on the page with Select tag
12. `puppeteer_hover`: Hover an element on the page
13. `puppeteer_evaluate`: Execute JavaScript in the browser console

Let me know if you need help with anything!
You: I want to list all the tables.
2025-09-24 17:31:15,839 - INFO - HTTP Request: POST https://api.groq.com/openai/v1/chat/completions "HTTP/1.1 200 OK"
2025-09-24 17:31:15,840 - INFO -
Assistant: ```json
{
  "tool": "list_tables",
  "arguments": {}
}
'```
You:

This is not actually calling the MCP tool list_table, because the triple backticks fences make the response unparsable by json.loads().

Therefore, I add 'without the triple backticks fences' explicitly in the system message. Now the LLM is outputting parsable json, and the tools can be called correctly and smoothly.

python main.py
You: What tools do you have?
2025-09-24 18:05:59,524 - INFO - HTTP Request: POST https://api.groq.com/openai/v1/chat/completions "HTTP/1.1 200 OK"
2025-09-24 18:05:59,525 - INFO - 
Assistant: I have the following tools:

1. `read_query`: Execute a SELECT query on the SQLite database
2. `write_query`: Execute an INSERT, UPDATE, or DELETE query on the SQLite database
3. `create_table`: Create a new table in the SQLite database
4. `list_tables`: List all tables in the SQLite database
5. `describe_table`: Get the schema information for a specific table
6. `append_insight`: Add a business insight to the memo
7. `puppeteer_navigate`: Navigate to a URL
8. `puppeteer_screenshot`: Take a screenshot of the current page or a specific element
9. `puppeteer_click`: Click an element on the page
10. `puppeteer_fill`: Fill out an input field
11. `puppeteer_select`: Select an element on the page with Select tag
12. `puppeteer_hover`: Hover an element on the page
13. `puppeteer_evaluate`: Execute JavaScript in the browser console

Let me know if you'd like to use any of these tools!
You: I want to list all the tables.
2025-09-24 18:06:08,181 - INFO - HTTP Request: POST https://api.groq.com/openai/v1/chat/completions "HTTP/1.1 200 OK"
2025-09-24 18:06:08,181 - INFO - 
Assistant: {
 "tool": "list_tables",
 "arguments": {}
}
2025-09-24 18:06:08,182 - INFO - Executing tool: list_tables
2025-09-24 18:06:08,182 - INFO - With arguments: {}
2025-09-24 18:06:08,184 - INFO - Executing list_tables...
2025-09-24 18:06:09,957 - INFO - HTTP Request: POST https://api.groq.com/openai/v1/chat/completions "HTTP/1.1 200 OK"
2025-09-24 18:06:09,959 - INFO - 
Final response: The database contains the following tables:

* products

Let me know what you'd like to do next!
You: Show me the contents of the products table.
2025-09-24 18:06:25,490 - INFO - HTTP Request: POST https://api.groq.com/openai/v1/chat/completions "HTTP/1.1 200 OK"
2025-09-24 18:06:25,493 - INFO - 
Assistant: {
 "tool": "read_query",
 "arguments": {
 "query": "SELECT * FROM products"
 }
}
2025-09-24 18:06:25,494 - INFO - Executing tool: read_query
2025-09-24 18:06:25,495 - INFO - With arguments: {'query': 'SELECT * FROM products'}
2025-09-24 18:06:25,505 - INFO - Executing read_query...
2025-09-24 18:06:27,951 - INFO - HTTP Request: POST https://api.groq.com/openai/v1/chat/completions "HTTP/1.1 200 OK"
2025-09-24 18:06:27,954 - INFO - 
Final response: The products table contains the following data:

| id | name | price |
| --- | --- | --- |
| 1 | Widget | 19.99 |
| 2 | Gadget | 29.99 |
| 3 | Gizmo | 39.99 |
| 4 | Smart Watch | 199.99 |
| 5 | Wireless Earbuds | 89.99 |
| 6 | Portable Charger | 24.99 |
| 7 | Bluetooth Speaker | 79.99 |
| 8 | Phone Stand | 15.99 |
| 9 | Laptop Sleeve | 34.99 |
| 10 | Mini Drone | 299.99 |
| 11 | LED Desk Lamp | 45.99 |
| 12 | Keyboard | 129.99 |
| 13 | Mouse Pad | 12.99 |
| 14 | USB Hub | 49.99 |
| 15 | Webcam | 69.99 |
| 16 | Screen Protector | 9.99 |
| 17 | Travel Adapter | 27.99 |
| 18 | Gaming Headset | 159.99 |
| 19 | Fitness Tracker | 119.99 |
| 20 | Portable SSD | 179.99 |

Let me know what you'd like to do next!
You:

How Has This Been Tested?

Tested with the default LLM model with default parameters in the example:

url = "https://api.groq.com/openai/v1/chat/completions"
headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {self.api_key}",
}
payload = {
    "messages": messages,
    "model": "meta-llama/llama-4-scout-17b-16e-instruct",
    "temperature": 0.7,
    "max_tokens": 4096,
    "top_p": 1,
    "stream": False,
    "stop": None,
}

Breaking Changes

No breaking changes.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

Checklist

  • I have read the MCP Documentation
  • My code follows the repository's style guidelines
  • New and existing tests pass locally
  • I have added appropriate error handling
  • I have added or updated documentation as needed

@yukuanj yukuanj requested review from a team and felixweinberger September 24, 2025 10:10
@felixweinberger felixweinberger added bug Something isn't working documentation Improvements or additions to documentation labels Sep 24, 2025
felixweinberger
felixweinberger requested changes Sep 24, 2025
View reviewed changes
Copy link
Contributor

@felixweinberger felixweinberger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @yukuanj thank you for this contribution!

I tested out and verified that the example you show is indeed broken in the same way for me - however, checking out your branch here doesn't consistently resolve the issue for me. Sometimes it works, sometimes it doesn't.

It still ends up wrapping the tool call in backticks - to be less brittle, I think we'd want to update process_llm_response in this file instead to handle the case when json ... is sent back?

@felixweinberger felixweinberger added the needs more work Not ready to be merged yet, needs additional changes. label Sep 24, 2025
@yukuanj
Copy link
Contributor Author

yukuanj commented Sep 24, 2025

@felixweinberger Thank you so much for reviewing and testing out this pull request.

I strongly agree with your point that we should update process_llm_response in this file, instead of solely relying on prompting LLMs.

What I changed

We only attempt a tool call when the response is a single JSON object, optionally wrapped in a fenced code block.
Both of these forms are supported:

  1. json {...}
  2. {...}
    If there’s any extra text before/after the JSON, we treat it as a normal reply (no tool execution).

How it works

A small regex strips a single fenced block (with or without the word 'json') and then try json.loads().

Example session

python main.py
You: What tools do you have?
2025-09-24 23:23:21,106 - INFO - HTTP Request: POST https://api.groq.com/openai/v1/chat/completions "HTTP/1.1 200 OK"
2025-09-24 23:23:21,107 - INFO - 
Assistant: I have the following tools:

1. `read_query`: Execute a SELECT query on the SQLite database
2. `write_query`: Execute an INSERT, UPDATE, or DELETE query on the SQLite database
4. `create_table`: Create a new table in the SQLite database
5. `list_tables`: List all tables in the SQLite database
6. `describe_table`: Get the schema information for a specific table
7. `append_insight`: Add a business insight to the memo
8. `puppeteer_navigate`: Navigate to a URL
9. `puppeteer_screenshot`: Take a screenshot of the current page or a specific element
10. `puppeteer_click`: Click an element on the page
11. `puppeteer_fill`: Fill out an input field
12. `puppeteer_select`: Select an element on the page with Select tag
13. `puppeteer_hover`: Hover an element on the page
14. `puppeteer_evaluate`: Execute JavaScript in the browser console

Let me know if you need help with anything!
You: I want to list all the tables.
2025-09-24 23:23:25,922 - INFO - HTTP Request: POST https://api.groq.com/openai/v1/chat/completions "HTTP/1.1 200 OK"
2025-09-24 23:23:25,925 - INFO - 
Assistant: ```json
{
  "tool": "list_tables",
  "arguments": {}
}
`` `
2025-09-24 23:23:25,926 - INFO - Executing tool: list_tables
2025-09-24 23:23:25,927 - INFO - With arguments: {}
2025-09-24 23:23:25,934 - INFO - Executing list_tables...
2025-09-24 23:23:27,664 - INFO - HTTP Request: POST https://api.groq.com/openai/v1/chat/completions "HTTP/1.1 200 OK"
2025-09-24 23:23:27,667 - INFO - 
Final response: The database contains the following tables:

- products

Let me know if you need help with anything else!
You:

Looking forward to your feedback!

@yukuanj
Copy link
Contributor Author

yukuanj commented Sep 30, 2025

@felixweinberger Hi Felix! I’ve updated the code so that it now passes the checks. Could you please take a look when you get a chance?

felixweinberger
felixweinberger approved these changes Sep 30, 2025
View reviewed changes
Copy link
Contributor

@felixweinberger felixweinberger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for making the changes, lgtm!

@felixweinberger felixweinberger removed the needs more work Not ready to be merged yet, needs additional changes. label Sep 30, 2025
@felixweinberger felixweinberger merged commit c81ff64 into modelcontextprotocol:main Sep 30, 2025
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@felixweinberger felixweinberger felixweinberger approved these changes

Assignees
No one assigned
Labels
bug Something isn't working documentation Improvements or additions to documentation
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@yukuanj @felixweinberger